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Final  Report  on  ARPA  Grant  Number  RF49620-91-C-0079 


A.  R.  Conn 

September  29,  1995 


Dear  Neal  Classman 

Below  is  the  final  report  on  ARPA  Grant  Number  RF49620-91-C-0079. 

1  PREAMBLE 

I  would  like  to  begin  by  stating  at  the  outset  how  grateful  I  have  been  to  receive  this  grant.  The 
grant  served  as  an  extremely  useful  catalyst  to  pursue  applications  of  optimization  to  manufac¬ 
turing  problems  and  I  have  every  intention  of  continuing  this  research. 

The  application  of  our  optimization  research  to  practical  manufacturing  problems  has  pre¬ 
sented  considerable  challenges  in  the  current  economic  environment.  Many  companies  have  been 
involved  in  the  process  of  restructuring  (including  IBM),  and  manufacturing  divisions  have  been 
hit  particularly  hard.  Some  manufacturing  facilities  have  already  been  dosed,  while  others  must 
either  meet  ambitious  new  production  targets  or  themselves  face  dosure.  Our  work  with  the 
manufacturing  divisions  reflects  this  uncertainty:  one  of  our  projects  was  terminated  by  the  do- 
sure  of  the  faciUty;  a  second  project  was  terminated  because  it  was  deemed  no  longer  critical 
to  meeting  the  plant’s  goals.  Moreover,  understandably,  this  is  an  espedaUy  difficult  time  to  be 
soliciting  cooperation  from  the  manufacturing  sector.  Moreover,  even  in  ideal  times  it  would  be 
\mrealistic  to  assume  that  we  would  awfliieve  our  ultimate  goal  (seeing  the  solution  implemented 
in  the  operation  of  the  manufacturing  site)  in  more  than  a  few  cases. 

Nevertheless,  we  have  been  working  on  real  problems,  collecting  real  data  and  have  established 
genuine  cooperation  with  manufacturing  personnel.  In  some  cases  it  is  dear  that  we  have  had 
significant  success  in  influencing  a  manufacturing  process  and  in  others  we  expect  similar  results, 
given  time. 

Moreover,  some  very  substantial  research  was  carried  out.  In  particular,  both  LANCELOT 
and  CUTE  have  international  reputations  and  I  am  very  confident  that  in  the  future  they  will 
have  some  high  profile  roles  to  play  in  the  manufacturing  sector.  The  derivative  free  work  is  also 
very  promising  and  enthusiasm  has  been  expressed  by  the  highest  executive  levels  in  research  at 
both  the  BOEING  and  IBM  corporations. 

I  certairdy  hope  to  apply  for  ARPA  funding  again  in  the  future. 
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to  solve  large-scale  nonlineax  optimizatioa  problems,  tbe  development  of  methods  of  optimization 
that  do  not  require  derivatives  and  are  relatively  insensitive  to  noise,  and  the  application  of  these 
algorithms  to  problems  in  manufacturing. 

We  achieved  our  three  main  objectives  in  the  development  of  algorithms; 

Completion  of  CUTE  (Constrained  and  Unconstrained  Test  Environment). 

This  test  environment  is  a  significant  contribution  to  both  researchers  and  practitioners. 
Researchers  can  use  the  provided  tools  to  build  interfaces  to  their  own  nonlinear  opti¬ 
mization  software  and  compare  its  performance  against  several  other  packages  on  a  large 
database  of  test  problems.  Practitioners  can  state  their  problems  in  the  standard  input 
format,  and  then  apply  several  different  packages  to  find  the  best  solution.  The  standard 
input  format  relieves  the  practitioner  of  repeatedly  inputting  each  problem  according  to  the 
specific  input  requirements  of  each  optimization  package.  This  software  has  been  accessed 
world-wide  by  more  than  two  thousand  users  and  for  some  has  become  a  de  facto  standard 
for  testing.  It  has  been  pubhshed  in  Bongaxtz  et  al.  (1995b)  and  has  already  been  quite 
widely  cited. 

With  the  completion  of  CUTE,  we  have  begun  the  numerical  testing  required  to  compare 
LANCELOT  and  MINOS,  which  are  the  two  foremost  large-scale  nonlinear  optimization 
packages.  Once  we  have  completed  the  generation  of  the  test  results,  we  hope  to  identify 
the  relative  strengths  and  weaknesses  of  the  nackages  on  broad  classes  of  nroblems.  As  with 


CUTE,  the  knowledge  gained  in  this  comparative  study  should  be  useful  to  researchers  and 
practitioners  alike.  Eesearchers  can  benefit  from  our  results  as  they  develop  new  algorithms, 
and  practitioners  will  know  which  software  package  is  best  suited  to  solving  their  problems. 

Testing  of  LANCELOT  .  - 

We  have  corhpleted  intensive  numerical  testing  of  LANCELOT  on  more  than  nine  hundred 
test  problems.  These  tests  helped  in  identifying  some  enhancements  to  the  algorithm,  and 
also  form  the  basis  for  our  comparison  of  LANCELOT  with  MINOS  in  Bongartz  et  al. 
(1995a).  Thus,  for  example,  our  results  where  the  number  of  inequalities  greatly  exceeded 
the  number  of  original  variables  motivated  our  analysis  in  Conn  et  al.  (1994b).  The  testing 
also  resulted  in  our  investigation  of  penalty  parameters  for  each  constraint  and  motivated 
a  separate  treatment  for  linear  constraints  in  Conn  et  al  (1993a)  and  Conn  et  al.  (1993b) 
that  is  to  be  published,  in  Conn  ef  al.  (1995c). 

Derivative  free  optimization 

In  the  application  area  there  is  a  real  need  for  methods  of  optimization  that  do  not  require 
derivatives.  For  example,  so-called  pattern  search  methods  such  as  Nelder  and  Mead  (1965) 
are  stiU  popxdar  with  practioners  although  significantly  better  methods  exist.  Moreover, 
because  of  the  nature  of  the  applications,  such  methods  should  be  able  to  handle  noise.  Mo¬ 
tivated  by  some  work  of  Powell  (see  Powell,  1994a  and  Powell,  1994b),  we  explored  the  use 
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of  multivariate  iaterpolation  techniques  in  the  context  of  such  methods  for  unconstrained 
optimization.  We  used  low  order  polynomial  models  in  a  trust  region  framework.  A  proto 
type  algorithm  exists  in  MATLAB  4.1,1  (see  The  Mathworks  Inc,  1992)  and  is  being  used 

in  S,  design  of  experiments  environmen't  io  ai tempi:  lo  optimize  tke  result  of  a  complex  and 
costly  numerical  procedure  required  in  the  anadysis  of  the  in  tight  vibration  of  a  helicopter 
rotor.  This  is  to  appear  in  Conn  and  Toint  (1995). 

In  applying  nonlinear  optimization  algorithms  to  problems  in  manufacturing,  we  have  tried 
to  find  practical  problems  faced  by  manufacturing  engineers  that  are  nonlinear  and  preferably 
of  generic  interest.  Our  objective  has  been  to  solve  problems  that  cannot  be  solved  adequately 
by  existing  methods.  V/e  have  had  some  success  throughout  the  duration  of  this  grant,  and 
we  are  hopeful  that  the  work  on  some  of  our  current  problems  will  make  useful  contributions 
to  manufacturing.  Below  we  list  the  applications  on  which  we  have  worked,  together  with  the 
manufacturing  site: 

-  batch  sizing  and  scheduling  the  assembly  and  test  of  hard  disks  (IBM  Rochester) 

•  maximizing  the  throughput  of  the  photolithography  sector  of  a  semiconductor  manufacturing 
line  (IBM  Burlington) 

-  identifying  the  region  of  feasible  solutions  in  manufacturing  design  problems  (Dassault 
France) 

•  minimizing  the  maximum  delay  in  small  transistor  circuits  (IBM  Kingston) 

•  tuning  circuits  (IBM  Austin) 

-  quality  control  and  testing  of  components  on  computer  boards  (IBM  Poughkeepsie) 

•  Optimization  of  the  design  of  complex  systems  in  the  aeronautical  industry  (Boeing  Corpo¬ 
ration) 

In  addition  to  the  applications  above,  we  have  developed  algorithms  to  solve  several  difficult 
classical  applications  problems  in  optimization: 

-  location- adlocation  and  related  problems 

-  macro-cell  placement  problems 

-  discontinuous  problems,  including  fixed  charge  problems,  nonlinear  robust  regression,  cutting 
stock  problems,  etc. 

-  problems  with  noise  and  proljlems  where  derivatives  are  unavailable 

These  algorithms  offer  significant  improvements  over  existing  methods. 

The  algorithmic  and  applications  work  combined  have  yielded  twenty  publications  that  have 
benefit  ted  from  and  acknowledge  ARPA  support. 

We  plan  to  continue  work  on  most  of  the  applications  Listed  above,  with  a  few  exceptions.  In 
particular,  the  focus  of  the  work  with  IBM  Burlington  has  shifted  from  optimizing  the  through- 


put  of  just  the  photolithograpliy  sector  to  modeling  the  entire  semiconductor  line  with  a  view 
to  reducing  cycle  time.  Their  current  priorities  do  not  encompass  optimization  so  this  project  is 
unlikely  to  continue  in  the  near  future.  In  addition,  the  IBM  Rochester  ADSTAR  manufacturing 
facility  has  been  closed,  and  so  work  on  the  batch  sizing  and  scheduling  problem  is  currently 
stopped.  If  possible,  we  Will  continue  this  research  with  a  similar  manufacturing  facility  in  IBM 
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SiE  Jos«,  More  importMUy,  we  continue  to  sctiyely  develop  teal  applications  in  tk  mamfec. 

turing  sectoi . 

We  now  give  more  details  an  the  research  outlined  above. 


3  Development  of  Algorithms 

3.1  tiTJTf;  (Constrained  and  Unconstrained  Testing  Environment) 

Tte  probl«m  of  inputtiag  nonlincjar  problems  in  thousands  of  variables,  inaintaining  a  database 
of  problems  and  being  able  to  teal  diiTerenl  packages,  all  of  which  expect  their  own  partintlar 
form  of  input,  is  an  important  and  nontrivial  task.  W«  have  developed  tools  to  probe  the  data 
set,  aid  m  the  evabiatiou  of  results  and  to  assist  in  interfacing  the  etandard  input  format  for 
LANCELOT  to  other  packages.  These  took  have  been  written  as  production  soflwaie  that  can 
be  mstalled  automatically  and  are  cunently  available  in  the  public  domain.  In  particular,  we  have 
an  interface  that  will  take  any  problem  input  intended  for  LANCKI.OT  and  preprocess  it  into  a 
suitable  format  lor  ii.se  with  MTNOS  Since  MINOS  and  LANCELOT  are  the  two  most  suitable 
production  software  packages  available  for  large-scale  nonlinear  optimization,  this  interface  will  be 
very  useful  to  many  researchers.  In  addition  we  have  Inlerfaces  lo  several  HARWELL  subroutines 
foi  yplimizatiou,  the  unconstrained  and  tensor  minimizers  of  Bobby  Schnabel,  and  IBM’s  OST,. 
Since  its  release,  oux  software  has  been  ttp-ed  worldwide.  The  .set  of  test  problems  Is  undoubtedly 
the  most  evten.sive  database  of  nonlinear  programming  problems  and  the  ability  to  readily  test 
alternative  optimization  software  on  problems  written  is  this  format  is  likely  to  be  invaluable  for 
algorithmic  devdopmeat  k  the  fidJ. 

Following  its  tdease.  we  have  worked  on  mstahation  script.-;  for  new  platforms  and  interfaces 
for  additional  optimization  packages,  induding  an  interface  to  NPSOL  and  interior  point  tech¬ 
niques.  Moreover  we  have  interface  tools  with  MATLAB,  (The  Mathworks  Inc,  1992).  Since 
^LA.TLAB  is  frequently  used  for  prototyping  algoritlmis,  this  should  be  especially  useful  for  re- 
searchers  in  optimization. 

This  research  was  recently  published  in  Bongartz  p.f.  nl.  (199.5bj 
3,2  Testing  and  Enhancement  of  LANCELOT 

Much  of  Our  work  in  this  area  has  been  based  upon  leaiiiiiig  from  our  extensive  numerical  expe¬ 
rience  to  date.  Thus  we  have  recognized  the  importance  of  accounting  for  the  speaal  structure 
of  slack  variables  and  now  have  an  adequate  approach  that  motivatsK  a  full  implementation.  The 
details  are  given  in  Conn  et  al.  (1994b). 

We  have  considered  the  implications  of  various  formulations  of  a  peaticular  group  partial 
separable  structure  and  have  been  able  lo  show  how  rdatively  simple  criteria  can  be  remarlably 
effective  at  improving  a  decomposition.  This  work  was  published  in  Conn  ef  al.  (19y4a).  More¬ 
over,  we  have  been  able  to  generalize  the  idea  of  a  single  trust  region  to  exploit  the  .structure 
inherent  in  partial  .separability.  Thi.s  work  appears  in  Conn  et  al.  (I995d). 

In  the  context  of  interior  point  methods  it  ia  not  dear  how  to  exploit  a  suitable  trust  re- 
gjon  modd  and  thus  we  need  to  investigate  both  this  and  line  seaidi  approaches.  Much  of  our 
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discontent  with  the  augmented  Lagrangian  framework  has  been  the  result  of  our  numerical  ex¬ 
perience  with  linear  constraints  and  this  has  led  us  to  consider  analogous  methods  for  interior 
point  techniques  in  addition  to  the  treatment  of  linear  constraints  separately.  This  resulted  in 
the  publications  Conn  et  al.  (I995e),  Conn  ei  al.  (I995f),  Conn  et  al.  (1994c),  and  Conn  et  al 
(1995a). 

We  are  also  using  our  numerical  tests  to  accumulate  numerical  data  concerning  the  efficacy  of 
various  preconditioner  updating  techniques  and  the  handling  of  degeneracy,  for  example.  Besides 
the  ongoing  work  (  see,  Bongartz  et  al,  1995a  and  Conn  and  Toint,  1994  -  these  projects  tend 
to  be  somewhat  long  term)  the  results  concerning  LANCELOT  alone  have  been  pubHshed  in 
Conn  et  al  (1988)  and  Conn  et  al  (1995b). 

3.3  Comparison  of  LANCELOT  and  MINOS 

For  general  large-scale  nonlinear  optimization  problems,  only  two  production  level  software  pack¬ 
ages  are  readily  available:  LANCELOT  and  MINOS.  Considering  that  MINOS  has  been  around 
since  the  70’s  and  is  well-known,  it  is  amazing  that,  with  the  exception  of  linear  programming 
results  (for  which  it  was  not  explicitly  designed),  its  numerical  performance  has  not  been  exten¬ 
sively  reported  in  the  literature.  The  linear  programming  performance  was  reported  when  the 
initial  comparisons  with  Karmarkar’s  interior  point  approach  were  made,  presumably  because  of 
the  lack  of  access  to  other  commercial  linear  programming  packages.  We  can  only  assume  that 
the  results  with  large-scale  nonlinear  problems  were  not  reported  because  of  the  lack  of  other 
available  codes  and  because  of  the  amount  of  effort  required  to  create  a  suitable  environment  for 
such  testing.  With  the  completion  of  CUTE,  we  have  begun  comparing  the  performance  of  these 
two  packages  on  a  set  of  test  problems  that  suitably  characterize  the  available  nonlinear  program¬ 
ming  database.  Since  some  of  the  test  problems  are  quite  large  (more  than  15,000  variables), 
generating  aU  the  test  results  is  an  extensive  task.  From  these  results,  we  hope  to  identify  the 
relative  strengths  and  weaknesses  of  these  packages  on  broad  classes  of  problems  (e.g.,  primarily 
linear  problems,  highly  nonlinear  problems,  etc.).  Once  again,  we  should  not  underestimate  the 
general  level  of  usefoiness  of  results  of  this  type.  We  are  fortunate  in  that  we  are  able  to  produce 
a  report  jointly  with  one  of  the  authors  of  MINOS  (Michael  Saunders). 

3.4  Derivative  free  Optimization 

We  consider  the  problem  of  minimizing  an  objective  function  whose  value  is  determined  by 
measuring  some  quantity  in  the  real  world.  This  measure  may  be  of  a  physical  nature  (for 
instance,  the  depth  of  a  certain  layer  in  geophysical  exploration)  or  be  related  to  other  contexts. 
We  focus  on  the  case  where  there  are  no  constraints  on  the  problem  variables  although  the 
generalization  to  simple  bounds  is  quite  straightforward.  Moreover,  since  the  proposed  method  is 
derivative  free,  one  might  want  to  handle  constrained  problems  using  an  exact  penalty  function. 

By  extension,  we  also  consider  unconstrained  optimization  problems  whose  objective  function 
is  the  result  of  a  complex  and  costly  numerical  procedure,  possibly  involving  some  considerable 
noise  (due,  for  instance,  to  truncation  or  approximation  in  the  calculation  defining  the  objective). 


5 


MflR  8  ’96  li:38 


FROM  BUS./GOU’T.  RELATIONS  '  TO  912024047951 


PAGE. 007 


We  explore  tLe  uce  of  m\dtivaria,te  interpoktiyu  ledmiques  In  the  context  of  methods  for 

‘ATI M  i  1  i  *  t  t  •  ~ 


II) 


II 


nol 


d  tie 


oLie 


require  clcnvalives  of  the  objective  function.  A  new 
algorithm  ia  proposed  that  uses  quadratic  models  in  a  trust  region  framework.  The  algorithm  ic 
wustiuaed  to  require  few  evaluations  of  the  objective  function  and  is  desiped  to  be  relatively 
m.sensitive  to  noise  in  the  objective  function  values.  The  details,  including  numerical  results,  axe 
published  in  Conn  and  Tvlnl  (1995). 


3.5  Iterated  Subspace  Minimir-ntioii 

Ihe  most  common,  methods  for  unconstrained  minimization  either  determine  a  search  direction 
followed  by  a  liaesearch  or  use  the  trust-region  appxo^cli  (see,  for  example,  Dennis  and  Schnabel. 
1083).  In  the  former  <t  simple  model  of  the  underlying  objective  function  is  ronf;tn.i.cted  in 
order  to  determine  the  search  direction.  By  contrast,  in  the  latter  case,  an  approximate  minimizer 
of  the  model  within  a  restnr.teH  domain  (the  trust  region)  le  determined.  This  model  mnimizer 
is  then  used  as  a  prediction  of  the  actual  minimircr  of  the  tme  objective.  In  a  trust- region 
method,  success  of  this  process  is  measured  by  comparing  the  model  and  true  function  vaiuej;  at 
the  predicted  miiumizer.  In  Uneseaxch  methods,  the  true  function  is  used  to  e.st.ablish  a  step  size. 
Thus,  both  of  these  approaches  may  be  con.^sidered  to  perform  their  multi«< dimensional  worlc  with 
respect  to  a  model  whilst  probing  the  true  function  uni- dimensionally.  Of  course,  the  model  does 
make  use  of  the  true  function  and  perhaps  its  deilvatives  —  maybe  at  more  than  a  single  point. 

In  this  paper,  wc  Lake  the  view  that  the  above  schemes  are  quite  wasteful,  given  the  amount 
^^f^^tnation  that  may  have  been  accrued  during  the  (approYimate)  miniroization  of  the  modal. 
In  partiriilar,  the  model  may  have  been  sampled  in  a  number  of  potentially  interesting  directions, 
of  which  only  the  aggregate  direction  is  norm2Jly  considered  to  be  of  biguifLcance. 

We  also  believe  provided  function  and  derivative  values  are  inexpensive  to  compute 
relative  to  the  linear- algebra  costs,  an  (approximate)  low-dimensional  TniniTnization  is  a  relativ^y 
simple  calculation.  Indeed ,  we  feel  that  there  is  high-quality,  robust,  general  purpoco  eoftwarc 
readily  available  for  the  email  scale  unconctrained  mixiimization  problem,  and  that  such  softwaic 
is  normally  capable  of  solving  problems  of  modest  Jiiiicitsioiis say  up  to  100  variable  problems 
-  extremely  fast  on  current  workstations  provided  that  function  evaluation  is  cheap.  Ot  course 
there  are,  and  will  continue  to  be,  small-scale  problems  which  are  challenging,  because  they  are 
so  nonlinear  that  algorithms  implemented  in  fixed,  finite  precicion  arithmetic  arc  unsuccessful, 
but  in  our  experience  such  examples  occur  rarely  In  practice. 

Thus,  in  this  paper,  wc  piupusc  met-Iiods  which  aim  to  investigate  the  true  obiective  function 
in  a  space  larger  than  the  one-dimensional  space  which  is  normally  associated  with  linesearch  or 
trust-region  methods.  We  do  this  knowing  that,  .so  long  the  space  is  relatively  modest,  the 
^•PP'royimate  multi- dimensional  minimization  will  still  be  a  manageable  calculation.  Moreover,  by 
carefully  choosing  the  space  that  wc  investigate,  we  hope  to  reduce  significantly  the  llnea: -algebra 
costs  while  still  maintaining  globdl,  <uid  fast  asymplotic,  convergence. 
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4.1 


Batch- Sizing  and  Scheduling 


111  Wfflicr  ADSai  iiiittiij  tiitif  piMiiii  ih  (m 

from  microa  to  minis.  These  fdcs  were  manufactured  not  only  for  IBM  products,  iurlnding  PS /x’s, 
RS/OOOO’s  and  AS/dOO’s,  but  also  for  a  growing  OKM  marVet  The  components  required  to 
manufacture  the  liles  were  provided  by  outside  suppliers.  The  files  bdonged  to  three  families, 
where  some  families  contained  ccvcral  models;  models  were  diffeicutiated  by  capacity,  density. 


and  performance. 


Tlic  twumain  rnnetions  in  the  manufacturing  process  were  assembly  and  test  In  the  assembly 
area,  the  files  passed  through  three  stages,  where  each  stage  had  several  lines  and  all  lines 
ar.comTTioda.tpd  all  file  types.  A  set-up  time  was  required  to  switch  a  line  from  one  file  type 
(or  sometimoG,  from  one  file  family)  to  another.  At  dcinaiiil  levels,  the  manufacturiiig  line  was 
operating  dose  lu,  and  in  some  cases  beyond,  capacity.  The  testing  area  in  partial iar  wa.^  ha.ving 
difficulty  meeting  the  demand  for  certain  file  type.<;,  thereby  creating  backlogs  of  files  waiting  to  be 
tP-ste.d.  Tn  addition  to  these  capacity  shortages,  there  were  parte  ohortages  for  some  components, 
such  as  the  disks  and  actuators.  Thus  the  mzmufacturing  and  coiiipuuenl  resources  had  to  be 
carefully  managed  to  ensure  Rochester  met  its  weekly  demands  for  files,  or  at  least  satisfied  as 
much  of  this  demand  as  possible. 

in  cooperation  with  Rochester,  wo  were  constructing  am  optimisation  model  to  solve  this 
file  production  problem.  Ultimately,  the  modd  should  have  solved  both  the  batch-saing  and 
scheduling  problem.  The  batch-sidug  problem  would  have  decided  how  many  of  each  file  to 
produce  each  day,  while  the  scheduling  problem  would  have  determined  in  what  order  the  hatrhe.-; 
should  have  been  produced.  The  batch  fshp-s  were  to  be  chosen  to  meet  as  much  of  the  demand 
as  possible,  STibject  to  constraints  on  production  capacity  and  component  n'tuilnbilitics. 

We  completed  a  batch-sizing  model  of  the  etsscmbly  area  which  decided  not  only  the  quantity 
of  each  file  type  to  pioduc«  cadi  day,  but  also  allocated  this  production  to  particular  lines  within 
the  assembly  area.  We  also  built  a  preliminary  model  of  the  testing  area  to  as.se..ss  the  usage  of  the 
lest  facilities.  With  the  assembly  and  test  area  data  .supplied  by  Rochester,  we  generated  some 
preliminary  results.  The  batch-sizing  model  ac  colvcd  involved  approximately  2500  variables  and 
2000  constraints,  not  including  simple  bound  coastraiuts.  The  optimization  enabled  us  to  analyze 
several  potential  scenarios  and  in  particular  the  bottlenecks  were  clearly  indicated.  We  prepared 
an  internal  technical  report  describing  these  details.  Because  of  the  production  data,  this  internal 
report  is  confidential,  but  we  plan  to  prepare  a  report  for  the  public  domain  that  describes  the 
solution  methods  but  docc  not  include  confidential  data. 

The  following  table  gives  some  sample  oulpul  Jroin  the  batch-sizing  model  for  the  assembly 
ai’ea,  under  the  assumption  that  each  day  we  have  only  enough  inventory  of  each  component  to 
produce  one-fifth  of  the  weekly  demand, 

(To  avoid  rekasing  confidential  data,  w©  have  altered  the  numbers  in  this  and  the  subsequent 
tabic,  but  the  qualitative  results  are  genuine.) 

In  the  scenario  above,  the  output  of  the  assembly  area  is  constrained  by  the  limited  component 
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Model 

Hay  1 

Day  2 

Da; 

Ji 

Dai-  4 

Da^  5 

Total 

1 

90 

121 

no 

no 

no 

550 

2 

771 

771 

771 

771 

772 

3856 

3 

376 

376 

376 

376 

3/5 

1879 

4 

1?'i7 

1257 

1257 

1257 

1257 

6285 

5 

266 

266 

266 

266 

267 

1331 

6 

486 

486 

486 

486 

486 

2430 

7 

183 

179 

184 

186 

183 

915 

Daily  Productioa 

3438 

3456 

,  3450 

j  3452 

3450 

17246 

mventory.  If  we  assume  that  wc  have  enough  inventory  available  eadi  day  that  we  could  produce 
the  entire  weekly  demand  in  one  day,  we  get  the  following  results: 


Model 

Day  1 

Day  2 

Day  3 

Day  4 

Day  5 

Total 

1 

0 

0 

0 

0 

564 

564 

2 

1789 

618 

628 

443 

505 

3983 

3 

.  0 

187 

469 

702 

521 

1879 

4 

2817 

0 

62/ 

968 

1873 

6285 

5 

0 

0 

302 

611 

418 

1331 

6 

338 

1091 

1099 

564 

0 

3082 

7 

202 

460 

0 

0 

253 

915 

Daily  Production 

5146 

2356 

3125 

3278 

4134 

18039 

Under  this  bcenario,  the  oulpuL  of  the  assembly  area  increases  by  4.59%.  Now,  instead  of  being 
constrained  by  the  limited  component  inventory,  we  are  constrained  by  the  available  production 
time  on  one  of  the  assembly  lines.  This  is  just  one  example  of  the  what-if  scenarios  that  could 
be  run  on  the  batch-sizing  model. 

Unfoitunatciy,  at  this  point,  the  Rochester  manufacturing  facility  was  dosed  and  work  on 
this  problem  is  currently  stopped.  Rochester’s  ADSTAR  manufacturing  operations  have  been 
consolidated  with  a  similar  facility  in  IBM  San  Jose',  and  we  hope  to  continue  our  research  with 
this  site. 

The  model  ac  firct  propooed  to  Rochcctcr  accumed  that  production  was  scheduled  on  a  build 
to-Oider  basis  and  that  capacity  would  not  always  be  sufficient  to  satisfy  demand.  In  this  envi¬ 
ronment,  the  objective  would  have  been  the  maximization  of  a  nonlinear  priority  function,  where 
tH?<;  function  Teflectp.d  the  demands  and  dne  dates  of  orders  for  the  various  files.  T.ater,  for  the 
purposes  of  the  initial  modal  at  least,  wo  ware  informed  that  production  was  actually  schedided 
on  a  build- to-ploji  basis,  with  relatively  constant  quantities  of  each  file  to  be  built  each  week. 
The  objeclive  of  inaxiim^^ing  ll^e  iioulineai-  priority  funcliun  was  therefore  replaced  with  the  ob¬ 
jective  of  maximizing  total  production,  which  is  a  linear  function.  The  resulting  batch- sizing 
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model  imder  this  objective  is  a  large  mixed  integer 


i.e. 


there 


III 


miabl«)  progranmm.g  problem  (i.e.,  the  objective  Wiop  and  the  conetramte  ate  <J1  linear) 
Thus,  m  It!  enrtent  form,  the  batch-siring  model  does  not  require  large-scale  nonlinear  program- 

i  iiii  11  ii  ii  Wd  1,  i„  ] jj  L  w^u, 

problems.  Nevertheless,  the  flexibility  of  our  model  allows  us  to  easily  return  to  the  objective  of 
priority  maximization,  if  it  turns  out  that  this  is  in  fact  the  better  approach. 


4.2  Stepper  Throughput  Problem  in  Photolithography 

PhotoHthography  is  the  process  by  which  a  microscopic  pattern  is  transferred  from  a  photo  mask 

to  a  material  layer  in  a  circuit.  For  example,  one  might  etch  into  a  silicon  dioxide  layer  on  the 
surface  of  a  silicon  wafer. 

A  simplifled  version  of  the  wafer  manufacturing  process  is  as  follows:  the  oxidized  wafer  is 
first  coated  with  a  layer  of  a  Hght  sensitive  material  called  photo  resist  and  then  exposed  to 
ultraviolet  light  through  the  photo  mask.  The  exposure  renders  the  photo  resist  insoluble  in  a 
developer  solution;  hence  a  pattern  of  the  photo  resist  is  left  wherever  the  mask  is  opaque.  The 
wafer  is  next  immersed  in  an  acid  solution,  which  selectively  attacks  the  silicoa  dioxide,  leaving 
the  photo  resist  pattern  and  the  silicon  substrate  unaffected.  In  addition,  MOS  circuit  elements 
typically  require  negative  and/or  positive  dopants  (for  example,  boron  and  phosphorus)  that 
are  implanted  via  ionization.  In  the  final  step,  the  photo  resist  pattern  is  removed  by  means  of 
auotlier  cliemical  treatment. 

The  cameras  for  this  process,  referred  to  as  steppers,  are  very  expensive  and  the  control  of 
the  lens  and  blades  is  an  inherently  slow  process.  A  typical  sequence  of  operations  on  the  stepper 
involves  loading  a  computer  program,  loading  a  reticle  (mask  for  one  or  perhaps  several  chips), 
setting  the  blades  (to  protect  the  rest  of  the  wafer),  loading  the  wafer,  doing  a  rough  alignment, 
refining  this  alignment,  and  then  stepping  (i.e.,  moving  from  chip  to  chip)  and  exposing,  with 
many  loops  involving  resetting  of  blades,  realignment,  etc.  Several  steppers  are  available  on  the 
process  line  but  not  all  chips  can  be  run  on  all  models.  Furthermore,  demand  often  exceeds 
capacity.  Various  options  (increasing  the  number  of  operators,  buying  new  masks  which  expose 
several  chips  simultaneously,  reducing  the  number  of  alignment  sites,  etc.)  are  available  to  in¬ 
crease  the  stepper  throughput,  but  these  options  can  be  costly.  The  problem  is  to  identify  those 
changes  which  yield  the  greatest  improvement  in  throughput  at  the  least  cost. 

We  built  two  versions  of  the  stepper  throughput  model,  namely  one  to  minimize  production 
cost  subject  to  constraints  on  production  time  and  demand,  and  one  to  minimize  production  time 
subject  to  constraints  on  production  costs  and  demand.  The  model  was  based  upon  empirical  data 
and  attempted  to  determine  the  best  investments  to  make  to  improve  throughput.  The  original 
model  included  only  one  tool  set,  namely  the  Nikon  steppers,  and  did  not  allow  the  possibility  of 
buying  new  masks  to  expose  several  chips  simultaneously.  The  model  was  first  extended  to  include 


the  option  of  purchasing  new  masks,  and  later  extended  to  include  a  representation  of  another 
tool  set.  With  the  additional  tool  set,  we  aimed  to  determine  how  production  of  the  wafers  should 
be  distributed  over  the  two  sets  to  increase  throughput.  However,  Burlington  recently  decided 
that  stepper  throughput  was  no  longer  their  biggest  problem  and  their  efforts  should  be  directed 
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towards  developing  a  model  of  tte  e: 


op| 


piiiiijiiiiii 


Work  oa  the  stepper  tliroughput  model  has  therefore  stopped  and  iaterest  has  shifted  to  the  more. 

global  situation  instead.  We  refer  to  this  new  model  as  the  ’bubble  pheaomenon’  and  give  details 
in  tic  next  section. 


Before  we  began  to  work  with  Burlington,  they  had  developed  a  nonlinear  funaiou  which 
calculated  the  throughput  of  the  Nikon  steppers  based  on  the  number  of  alignment  sites,  the 
number  of  blade  settings,  the  number  of  exposoics  per  wafer,  etc.,  but  they  did  not  know  how 
to  miudmize  this  function,  especially  if  constraints  were  imposed  on  the  allowable  ranges  for 
some  vaikbles.  To  ’analyze’  this  function,  they  set  all  but  one  of  the  variables  to  their  current 
values,  and  then  evaluated  the  throughput  function  for  several  different  values  of  the  remaining 
variable.  This  approach  told  Burlington  what  the  lirst-ordei  cifecLs  of  each  of  the  variables  were, 
bat  could  not  tell  them  what  the  best  overall  solution  would  be.  We  were  not  only  able  to 
majdinize  theii-  original  throughput  function,  but  we  were  also  able  to  include  additional  terms 
for  new  production  vanables  (such  as  purchasing  new  masks)  and  to  include  constraints  on  the 
production  va.riables.  It  should  bs  noted  that  the  throughput  function  is  inherently  nonlinear, 
and  therefore  linear  programming  techniques  would  not  be  appropriate  for  this  application. 

We  Wid  use  an  example  to  illustrate  the  results  provided  by  our  model.  Suppose  our  objective 
is  to  minimize  daily  production  cost  subject  to  satisfying  demand.  The  fonowing  table  ahows  the 
best  change.s  to  make  to  improve  stepper  throughput  as  the  daily  demand  is  incicescd; 


Daily  Demand 
1000  wafers/ day 

IlOU 

1200 

1300 


Change  to  Improve  Stepper  Throughput 

PuKiiasc  new  masks  lo  reduce  SPW  (number  of  shots 

per  wafer)  to  30. 

Purr.h^j5e  new  masks  to  reduce  SPW  to  25  (the  minimum 
posfiihk)  and  hire  additional  operators  to  act  as 
work  coordinators. 

Purchase  new  masks  to  reduce  SPW  to  25  and 
hire  as  many  additional  op<^rators  as  possible. 

Demand  can  no  longer  be  satisfied. 


(To  avoid  releasing  confidential  data,  we  hav^  alt.Ared  the  numbers,  but  the  qualitative  resultc 
axe  genuine.) 
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The  results  above  assume  all  productior,  is  nm  on  tb  Jf g^tippGrfl 

15  passes  and  at  most  five  of  these  passes  can  be  rim  on  the  Perkin  Elmers 
get  the  following  results; 


Ifcacliwi^wryQm 


Daily  Demand 
1400  wafers/day 

1500 


Change  to  linpiov^  Stepper  Throughput 

Run  11  pci53C3  on  Nikons,  4  peases  on  Perkin' Elmers. 

Also  purchase  new  masks  to  reduce  SPW  to  2b. 

Run  10  passes  on  Nikons,  5  passes  on  Perkln-Elmers. 
Purchase  new  masks  to  reduce  SPW  to  25  and  hire 
additional  operators. 


Run  10  passes  on  Nikoiis,  5  parses  on  Perkin- Elmers. 
Puichatie  new  masks  to  reduce  SPW  to  2b  and 
as  many  additional  operators  as  possible. 


lYUO 


1800 


Run  10  passes  on  Nikons,  5  peussca  on  Perkin- Elmers. 
Purchase  new  masks  to  reduce  SPW  lo  25  and  hire 
as  many  addliioual  operators  as  possible. 

Reduce  the  number  of  alignment  sites. 

Demand  ran  no  longer  be  satisfied. 


4.3  Bubble  Phenomenon 

The  Burlington  .‘;eTn.i conductor  manufacturing  line  in  Building  973  manufactures  O-inch  logic 
wafers.  The  engineering  ctaff  which  runs  this  line  is  currently  f^ced  with  the  challenge  of  reducing 
this  line’s  cycle  time  by  30%.  In  the  course  of  studying  the  flow  of  wafers  through  their  line.,  the 
<cifgiiieers  h^vc  discovered  that  this  flow  is  not  always  constant,  but.  sometinies  contains  significant 
Tjubbles’,  which  consist  of  ma.ny  wafers  at  one  gate  in  the  line.  These  bubbles  are  created  when 
one  gate  falls  behind  on  its  production,  usually  because  of  equipment  break  downs  at  that  gate. 
The  gates  most  likely  to  cause  these  bubbles  are  those  that  have  only  a  few  tools.  When  all  the 
tools  in  such  a  gale  are  operational,  the  capacity  significantly  exceeds  the  production  requirement 
With  even  one  breakdown,  however,  the  capacity  of  the  gate  falls  significantly  below  required 
levels.  For  example,  an  ion  implant  gate  with  three  took  may  run  at  120%  of  the  daily  going 
rate  when  all  three  took  arc  operational,  but  this  level  drops  to  80%  when  one  tool  goes  down. 
The  wafers  then  accumulate  in  front  of  the  ion  iiuplanters,  waiting  to  be  processed.  During  this 
liiue,  subsequent  gates  might  be  starved  for  parts,  and  they  too  will  he  idle  Once  all  the  ion 
knplanters  are  operational  again,  the  w^^fers  will  be  processed  at  a  rate  that  excccdc  the  capacity 
of  subsequent  gates  in  the  line.  The  waferc  will  again  accumulate  in  front  of  these  subsequent 
gates,  and  the  bubble  will  persist  in  the  production  line  fo.£  luauy  days. 

Burlington  would  like  to  model  the  babble  phenomenon,  and  use  the  model  to  identify  changes 
ill  the  line  that  would  help  the  wafers  flow  more  smoothly  through  the  line.  'RAdnring  the  WIP 
(work  in  process)  should  lead  to  a  reduction  in  the  line’ji  ryde  time. 
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I'o  bftgiTi ,  we  begaa  by  modeling  not  tie  fdl  O-ludi  semiconductor  manuiactiiTing  line,  but 
rather  a  mini  line  wHch  consists  of  fourteen  gates.  These  gates  were  chosen  by  Burlington  to  be 
representative  of  the  most  imnortant  gates  in  thf  ffilTImi  ' 


Burlington  now  need  to  ga.ttcr  the  necetibary  test  d^ta, 


Wn 


an 


al  and 


Burlington  already  has  a  detailed  simulation  model  of  their  semiconductor  manufactuiing 
line.  This  simulation,  however,  does  not  run  much  faster  than  real  time.  (We  do  not  know  the 
ratio  of  simulation  time  to  actual  production  time,  but  it  would  certainly  take  many  hours  to 
model  one  month  of  production.)  To  address  the  bubble  phenomenon  with  the  simulation  tool, 
an  informed  user  would  have  to  run  a  simulation  based  on  the  rurrent  production  line,  identify 
changes  that  might  improve  the  bubble  phenomenon,  run  a  simulation  on  the  changes,  use  the 
Simulation  output  to  identify  new  changes,  and  00  on,  until  a  set  of  changes  had  been  identifed 
that  would  alleviate  the  bubble  phenomenon.  Even  then,  the  identified  changes  would  not  likely 
be  the  best  possible  ones,  but  lalher  oidy  the  best  of  those  sim'ulated.  (What  ’best’  means  in  this 
application  must  be  decided  by  Burlington.  It  might,. for  eicample,  mean  the  cct  of  changes  which 
minimizes  WlP  subject,  to  budget  constr^ts.)  With  the  current  simulation  tool,  this  process 
could  easily  take  weeks.  We  expect  that  our  optimization  model  will  run  significantly  faster 
and  identify  the  best  improvements  to  the  line.  Unfortunately,  however,  Burlington’.*;  ci.trrent 
priorities  have  prevented  them  from  continuing  on  this  project  at  this  time. 


4.4  M^uiufacturing  Design  Problems 

Dassault  Prance  have  a  generic  set  of  manufacturing  design  problems.  Typically  their  designs  (for 
an  entire  aircraft  body,  for  caample)  involve  a  number  of  variables,  inequality  constraint.^  and 
equations.  The  number  of  variables  can  be  large,  can  include  some  logical  and  integer  variables 
and  at  least  some  of  the  relationships  can  be  highly  nonlinear.'-  What  they  desire  is  to  be  able  to 
obtain,  some  general  idea  of  the  entire  region  of  feasible  solutions  so  that  in  the  manufacturing 
process,  they  have  the  possibility  of  choosing  coiiipoiieiils  that  are  preferable  on  the  basis  of  other 
(subjective  and  objective)  criteria.  : 

To  date  we  have  demonstrated  that  we  are  able  to  develop  such  solutions,  and  indeed,  in  the 
case  Ot  the  design  of  an  aircraft  structure,  we  have  been  able  to  indicate  to  them  that  they  have 
not  been  solving  the  correct  problem,  because  of  inconsistency  in  the  data. 

Wc  arc  currently  modeling  a  complex  engine  gearing  system  for  which,  according  to  Dassault, 
no  solution  has  ever  been  determined. 

Eventually  they  would  like  to  incorporate  onr  techniques  into  a  generic  design  tool,  if  this  is 
possible. 

It  is  hard  to  be  more  definite  at  this  time  as  to  the  eventual  benefits  since  we  are  still 
at  a  relatively  early  stage.  However,  from  the  discussions  and  experiences  that  1  have  had 
with  Dassault,  Ftance  it  would  appear  that  we  are  solving  the  problems  much  faster  than  they 
expected,  which  is  very  important  from  the  point  of  view  of  incorporating  our  techniques  into  a 
generic  de.sign  tool 

It  should  be  stressed  that  these  problems  arc  inherently  very  uoulincai'  and  it  is  essential  that 
nonlinear  techniques  be  used. 
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On  another  front,  we  are  applying  the  derivative  free  optimization  algorithm  to  optimize 
complex  systems  using  both  design  of  experiments  models  and  methods  related  to  pattern  search 
techniques.  This  is  joint  work  with  Rice  University  and  the  Boeing  Corporation. 

Typical  applications  generate  very  expensive  function  evaluations,  often  determined  via  so¬ 
phisticated  simulation  packages.  Consequently  the  ‘outer  model’  provided  by  the  simulation 
does  not  readily  lend  itself  to  straightforward  nonlinear  optimization.  For  example  in  a  quasi- 
Newton  sequential  quadratic  programming  context  the  cost  of  determining  and  solving  successive 
quadratic  programs  would  be  prohibitive,  even  assuming  hist  derivatives  axe  available  from  the 
simulation.  However,  these  types  of  engineering  optimization  problems  arise  frequently. 

In  the  context  of  this  joint  project,  the  particular  problem  of  interest  is  optimization  of  the 
engineering  design  of  a  helicopter  rotor  blade  so  as  to  minimize  vibrations  for  specified  flight  con¬ 
ditions.  This  design  is  suffiaently  complex  and  expensive  so  as  to  make  the  use  of  mathematical 
models  imperative.  There  is  a  solid  basis  of  computational  fluid  dynamics,  structural  analysis 
and  control  theory  that  theoretically  allows  standard  nonlinear  optimization  to  take  place  but 
although  such  a  system  is  deterministic,  realistic  optimization  for  even  moderate  size  problems 
is  proiiibitively  expensive  and  slow. 

Consequently,  the  main  thrust  of  the  project  is  to  explore  approximate  models  over  which  it 
is  more  reasonable  to  optimize.  The  immediate  questions  that  arise  axe 

•  What  constitutes  a  suitable  approximation? 

•  What  is  an  appropriate  optimization  technique? 

•  What  represents  a  ‘solution*? 

Moreover  these  basic  questions  are  clearly  related  in  that  initially,  when  far  from  the  solution,  it 
would  be  efficient  to  have  a  relatively  crude  model,  optimization  technique  and  stopping  criterion. 
However,  ultimately,  when  in  the  neighborhood  of  a  solution,  one  could  expect  to  be  more  careful. 
Finally,  given  the  relative  accuracy  of  the  modeling  technique,  the  presence  of  noise  and  perhaps, 
the  absence  of  derivatives,  it  makes  no  sense  to  use  a  stopping  condition  that  aims  to  be  more 
accurate  than,  for  example,  the  noise  threshold. 

At  Boeing,  Paul  Frank  has  been  working  in  the  area  of  optimization  of  approximate  models 
over  the  past  year  and  it  is  clear  that  Boeing  intends  to  continue  with  this  project.  For  some 
time  John  Dennis  and  Virginia  Torczon  have  been  working  productively  with  the  multidisciplinary 
optimization  (MDO)  group  at  Boeing  Computer  Services.  Moreover,  my  colleagues  Katya  Mints 
(a  doctoral  student  at  Columbia  University  who  spent  the  summer  of  1995  at  T.  J.  Watson 
Research  Center)  and  Philippe  Toint  (University  of  Namur),  and  I  are  working  on  an  extension 
of  the  algorithm  cited  above  and  published  in  Conn  and  Toint  (1995).  Thus  this  collaboration 
involves  a  strong  commitment  accompanied  by  matching  and  complementary  skills  and  we  expect 
it  to  be  an  ongoing  project  with  significaint  applications. 

4.5  Circuit  Design  * 

This  encompasses  two  applications.  The  first  concerns  the  design  of  small  circmts  and  wa^  carried 
out  through  Philip  Strensky  of  the  T.J.  Watson  Research  Center. 
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We  are 


II 


and  few  ontpnts.  The  objective  is  to  make  the  circuit  very  fast.  In  our  initial  application  we 
used  a  simulator  to  run  some  standard  patterns  on  the  circuit  and  lucdsured  the  delay,  which 


is  a  function  of  the  tiansistOi  widlliS*  OUT  nhifif, 


111 


fifnrtion  of  the  widths,  subject  to  simple  bound  conctraints  on  the  widths.  We  foimulatcd  lliib  as 
a  nonJineax  programming  problem.  We  used  about  tweiit;y  widtkii,  although  this  could  increase. 
The  expense  in  solving  the  upiimi^iation  problem  is  in  the  simulation  (equivalent  to  a  tunctioTi 
evaluation).  Thus  we  fitted  a  quadratic  approximation  to  the  delay  function.  This  gave  us  a 
local  approxirnate  .mrface  and  we  then  iterated  on  the  approximations,  using  LANCELOT.  The 
results  were  used  with  success  in  the  IBM  Kiagston  plant.  We  would  like  to  consider  using  the 
simulator  directly  in  the  optlndzation  (without  fitting  a  quadratic  approximation),  but  we  then 
need  to  account  for  noise. 

In  general,  we  feel  there  is  Tniirh  scope  for  synthesklng  optimisation  techniques  with  cimu> 
latlon  systems  and  we  would  like  to  pursue  this  further.  There  has  been  some  interest  shown 
by  Burlington  Technology  Simulation,  who  use  a  sopiiistkated  siiuulalioii  system  in  the  design 
of  semiconductor  wafers.  This  is  like  a  much  bigger  version  of  the  Kingston  circuit  problem. 
However,  we  have  yet  to  meet  with  this  group  at  Burlington. 

The  algorithm  is  being  used  at  two  sites  in  the  USA  (Kingston  and  Austin)  and  one  site  in 
Germany  (Boeblingen). 

Previousij^  ’’opiimizaLion”  was  done  by  informed  users  making  mielligeat  guesses,  doing  sim¬ 
ulation  and  then  moving  around  in  the  design  space.  The  use  of  more  sophi.‘;t.icAtAd  uonlmepiT 
optimization  techniques  has  given  solutions  that  axe  10%  better  —  which  in  practice  is  a  great 


deal. 

More  recently  we  have  consldci  a  direct  exteusion  of  this  work  which  includes  analytic  ex¬ 
pressions  and  does  not  use  simulation.  It  turns  out  that  the  optimization  in  this  case  can  be 
more  eftidently  ca.TTied  out.  u.sing  MTNOS  and  a  direct  rouseqirence  of  this  work  Dr.  Strensky 
is  pursuing  the  idea  of  obtain  suitable  licenses  for  MINOS. 

Once  again,  I  hope  that  this  will  be  an  ongoing  project. 

The  second  application  wcw  coined  out  with  Chanda  Visweswaxiah  and  Ruud  Haring  of  the 
T  J.  Watson  Research  Center  and  involves  the  tuning  of  circuits.  Many  aremt  designers  spend  a 
great  deal  of  time  Tnanually  sizing  their  .srh.eTPa-tic.s  to  meet  delay,  power  and  area  targets.  Circuit 
simulatioix  is  often  carried  out  in  the  inner  loop  of  this  iterative  and  ’’optimization”  ic  typically  a 
manual  procesa.  Instead  we  have  incorporated  a  modified  version  of  LANCELOT  to  replace  the 
manual  process  with  real  optimization. 

The  results  to  date  with  a  prototype  research  version  have  been  very  encoiiTaging  and  we 
hope  that,  this  will  be  a  continuing  and  very  successful  application.  Should  that  be  the  case 
there  is  much  work  that  rcmainc  to  be  done  to  ensure  robustness  and  adequate  starting  and 
stopping  criteria  in  the  presence  of  noIsc.  Moicovei  we  expect  to  obtain  useful  feedback  from 
circuit  designers  who  will  be  trying  to  use  LANCELOT  to  design  real  and  large  circuits. 
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4.6  Quality  Control  and  Testing 

This  problem  concerns  the  testing  uf  boards  on  mainframe  madliiies  with  a  view  to  obtaining  the 

iOlllll«llllll|llllllill.Ui„J_LLJJ. 

Suppocc  one  has  a  board,  contaiEmg  many  component,  that  is  failing.  Typically  one  replaces 
a  subset  of  the  components  with  new  componenis  xo  correct  the  failure.  Since  the  components 
come  from  several  different  manuiactiirers,  we  would  lile  to  be  able  to  identify  the  source  of  the 
malfunctioning  component.  Due  to  cost  and  diagnoctic  constraints,  however,  the  exact  cause  uf 
the  failure  might  be  unknown  and  we  are  necessarily  lestxictcd  to  incomplete  observations  for 
much  of  the  data.  However,  it  is  essential  that  we  have  some  complete  information  as  a  result  of 
subsequent  testing.  Confronted  with  this  real  situation  we  would  like  to  erploit  all  the  availabk 
mlormation  in  an  optimal  way. 

The  optimisation  function  is  thus  to  maximize  a  likelihood  function  subject  to  simple  bounds 
on  the  variables.  The  problem  is  difficult  in  that  the  growth  in  the  number  of  variables  (because 
of  the  large  number  of  conditional  probabilities  involved)  is  rapid  as  the  number  oi  components 
in  the  subset  increases. 

To  date  we  have  optimized  a  simple  instance  and  we  have  explored  a  more  general  instance 
with  equally  encouraging  results.  With  B-  Reiser  of  the  Univeislty  of  Haifa,  Israel,  Betty  Schultz 
<Liid  myself  of  our  group  here  at  Watson  we  have  collected  our  results  in  Reiser  et  al.  (1994).  We 
would  like  to  generalize  this  framework  m  order  to  obtain  contidence  intervals  using  simubtion 
The  method  of  choice,  suggested  by  the  statistidaas,  is  to  use  the  bootstrap  techtuque,  Efron  and 
Tibchiiaiu  (1986),  for  obtaining  confidence  intervals.  This  would  require  repeated  optimization 
using  computer  simulated  data.  Depending  upon  the  direction  taken,  again  determined  by  the 
expertise  of  the  statisticians,  there  is  a  possibility  that  this  work  will  continue.  Eventually,  it 
is  hoped  that  the  approach  will  have  wide  application  to  quality  control  and  reliability  testing 
within  manufacturing. 

The  current  standard  practice,  when,  for  example,  a  board  coutainiug  k  componenis  fails 
with  a  probability  p,  is  to  assign  equal  probabilities  (i.e.,  p/k)  to  the  possibility  that  any  given 
component  fails.  The  proposed  approach  is  inherently  superior  and  moreover  it  is  evident  from 
our  current  experience  that  we  are  able  to  obtain  useful  solutions  to  problems  that  are  too  difficult 
to  analyze  otherwke. 

4.7  Lu  ca  lion- Ailo  ca  lion 

Location-allocation  problems  are  concerned  with  the  optimal  service  or  supply  of  a  set  ot  existing 
facilities  whose  locations  are  flved  and  known.  Service  of  the  existing  facilities  is  achieved  by  a 
set  of  new  facilities,  where  the  location  of  each  new  facility  must  be  determined  and  itc  output 
allocated  to  some  particular  subset  of  the  existing  facilities.  The  locations  and  allocations  of  the 
ucw  facilities  are  chuscxi  lu  luiiiiiuize  some  cost  function. 

Many  problems  can  be  characterized  as  location-allocation  problems,  inciuding  the  location 
oi  warehouses  or  factories  to  serve  custom ers;  the  location  of  pnhlic.  b/'lllties  such  ambulance 
depots,  fire  stations,  or  schools  to  serve  surrounding  neighborhoods;  the  optimal  layout  of  ma 
chines  in  a  plant,  or  the  addition  of  new  machines  to  existing  plant  layouts;  the  partitioning  into 
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STibgioups  of  circuit  modules  ia  VLSI  layout;  and  the  problem  of  cluster  analysis,  where  the  data 
points  are  treated  as  existing  facilities,  and  the  duster  centroids  as  new  facilities. 

We  have  completed  our  basic  algorithmic  work  on  the  uacapadtated  algorithm.  We  per¬ 
formed  extensive  numerical  tests  to  compare  our  algorithm  with  other  competitive  algorithms. 
These  tests  revealed  several  areas  where  our  algorithm’s  efficiency  could  be  improved,  and  these 
improvements  were  incorporated  into  our  current  software.  The  resulting  algorithm  is  not  faster 
than  alternating  methods,  although  its  solution  times  are  coinparable  with  alternating  methods 
and  faster  than  all  other  methods.  More  importantly,  however,  it  consistently  gives  better  quality 
solutions  than  all  other  methods  and  is  significantly  more  general  in  its  approach.  This  work  has 
been  published  in  Bongartz  et  al.  (1994). 

This  additional  flexibility  has  enabled  us  to  consider  the  generalization  to  the  capacitated 
problem.  We  have  completed  the  theoretical  work  to  extend  our  approach  to  indude  capacity 
and  demand  constraints  and  we  have  written  and  tested  a  preliminary  version  of  the  code  in 
MATLAB.  We  are  currently  looking  at  ways  to  improve  the  code  (starting  points,  degeneracy 
handling,  etc.) 

Our  previous  work  on  location-allocation  problems  enabled  one  of  us  (Ingrid  Bongartz)  to 
contribute  to  the  development  of  a  ’’logistics  planning  tool”  for  IBM’s  spare  parts  distribution 
system  in  the  United  States.  This  work  was  done  jointly  with  Management  Technologies  of  the 
IBM  Consulting  Group  in  Dahas,  TX,  and  was  not  funded  through  the  ARPA  grant.  The  client 
and  funder  was  IBM  U.S.  Logistics  in  Mechanicsbuig,  PA. 

The  problem  addressed  by  the  logistics  planning  tool  is  more  general  than  the  location- 
allocation  problems  considered  previously.  In  the  logistics  planning  problem,  the  facilities  to  be 
located  are  distribution  centers,  in  which  the  spare  parts  are  stored  and  from  which  they  are 
shipped  to  customers  whose  equipment  has  failed.  The  tool  determines  not  only  the  location 
of  the  distribution  centers  and  the  assignments  of  customers  to  these  centers,  but  the  inventory 
levels  for  each  part  at  each  distribution  center  and  the  mode  of  transportation  to  be  used  to 
transport  parts  to  each  customer.  The  demands  are  based  on  the  expected  failure  rates  of  the 
parts.  The  system  must  be  designed  to  satisfy  stringent  service  constraints,  which  stipulate  the 
percentage  of  replacement  parts  which  must  be  received  by  the  customer  within  certsdn  elapsed 
times  (e.g.,  4  hours  or  24  hours)  from  failure  diagnosis.  The  objective  is  to  minimize  the  overall 
costs  of  the  distribution  system,  including  the  costs  of  operating  the  distribution  centers,  owning 
and  handling  the  inventory,  and  transporting  the  parts  to  customers. 

Although  the  logistics  planning  tool  does  not  use  any  of  the  algorithms  we  developed  for 
location- allocation  problems,  our  expertise  allowed  us  to  quickly  understand  the  modelling  re¬ 
quirements  and  to  improve  the  initial  algorithms  developed  by  Management  Technologies.  The 
solutions  recommended  by  the  logistics  planning  tool  have  the  potential  to  save  IBM  U.S.  Lo- 
pstics  millions  of  dollars.  This  work  reaffirmed  our  belief  that  location-allocation  problems  have 
important  practical  applications  and  aure  therefore  a  worthwhile  topic  of  research. 

In  a  problem  related  to  the  location- allocation  problem,  Michael  Overton  of  the  Courant 
Institute,  New  York  University  (whilst  visiting  Watson  during  the  summer)  and  I  extended  our 
earlier  work  on  minimizing  the  sum  of  Euclidean  norms.  We  define  a  primal-dual  interior-point 
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method  that  Js  easy  to  motivate  and  implement.  Furthermore,  it  is  very  robust  and  converges 
very  rapidly  even  when  many  norms  In  the  stmnuand  are  zero.  The  key  observation  which 


cuiuplemeniarity  condition.  The  method  is  apparently  as  robust  a.s  primal-dual  interior-point 
methods  for  hnear  programming,  in  the  sense  that  almost  any  choice  of  stepkngth  rule  and 
almost  any  rule  for  reducing  the  centering  parameter  wJl  usually  woik,  including,  in  most  but 
not  all  cases,  just  setting  the  steplength  to  one  and  the  centering  parameter  to  zero.  A  prototype 
MATLAB  code  has  been  used  to  compute  optimal  Steiner  trees  given  their  topology  and  shoidd 
be  useful  in  many  other  applications  also.  Tn  particular,  although  we  originally  intended  to  just 
publish  an  internal  report,  there  has  been  enough  external  interest  in  the  results  that  we  axe  now 
motivated  to  write  a  full  paper.  Two  applications  that  we  aic  awaie  of  that  have  made  direct 
use  of  our  results  are  Li  plasticity,  (Anderson  and  Christiansen,  1995)  and  image  reconstruction, 
(Chan  er  <2/.,  1995). 

4.8  Vision  Sensor  Planning  Problem 

The  vioion  sensor  planning  problem  2Lri5cs  in  robotic  vision  tasks.  It  refers  to  the  calculation 
of  paidiiicler  values  for  vision  sensors  that  allow  automated  machines  to  perforin  predetermined 
functions,  Steven  Abrams  and  Peter  Allen,  oi  Columbia  University,  and  Konstantinos  Tarabanis, 
of  Manufacturing  Research  at  IBM  Thomas  J.  Watson  Research  Center,  have  formulated  the  sen¬ 
sor  planning  problem  as  a  nonlinear  optimization  problem.  This  problem  consists  of  deteimiiuiig 
the  sensor’s  lens  center,  viewpoint  d^iJ  focal  setting  in  order- to  maximize  the  sum  of  several 
diiferent  feature  detectability  criteria.  These  criteria  are  also  the  constraints  ot  the  optimization 
problem  They  ensure  that  all  features  on  the  object  to  be  viewed  are  visible  (i.e.,  not  occluded), 
axe  in  focus,  axe  within  the  eencor’c  field  of  view,  and  appear  large  enough  on  the  image  plane  to 
be  seen  by  the  vision  system. 

At  the  outset  we  worked  together  will  Abrams  and  Tarabanis  to  state  this  problem  in  SIF 
format  and  then  solve  it  using  LANCJELOI'.  Because  some  of  the  constraints  axe  nonsmooth, 
LANCELOT  had  difficulty  in  converging  to  optimal  solutions.  Unfortunately  this  work  did  not 
progrcoG  beyond  the  initial  numerical  experiments  although  we  had  some  interesting  idea  for 
haudliiig  the  uonsmoolli  iiaLaie  uf  the  problem.  The  group  in  Metnufacturing  Research  that  was 
investigating  sensor  planning  problems  was  sold  by  IBM. 

4.9  Layout  Problems 

Th©  aim  of  this  work  is  to  develop  a  projected  penalty  function  technique  to  solve  the  macro- 
cell  placement  problem,  implement  the  results  in  a  working  algOiithui  and  couipoie  the  algorithm 
with  methods  currently  being  employed.  It  constitutes  joint  work  with  David  Mates  (MacDonald, 
Dettwiler  and  Associates  Ltd)  and  Anthony  Vannelli  (University  of  Waterloo)  and  >s  the  major 
body  of  the  doctoral  thesis  of  th©  former. 

The  macro- cell  placement  problem  ic  the  problem  of  placing  irregularly  shaped  cells  (in  our 
case  we  consider  only  rectangular  shapes)  on  a  surface  such  that  the  distance  between  those  cells 
that  are  connected  (specified  before  hand)  is  minimized.  There  are  many  important  applications 
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of  this  problem  in  engineering  design.  Some  examples  are  printed  circuit  board  design,  very  large- 
scale  integration,  operations  research  location-allocation  problems  and  plant  layout  problems. 
This  is  the  first  time  that  this  problem  has  been  modeled  exactly  and  solved  with  a  piecewise 
linear  mathematical  program. 

To  accompany  the  projection  algorithm,  a  computer  graphics  package  was  developed  to  display 
the  current  layout  for  any  given  iteration.  This  package  makes  interpretation  of  the  results  much 
easier  as  well  as  allowing  the  user  to  manipulate  the  layout  to  aid  in  the  choice  of  starting  points 
or  to  restart  the  problem  in  the  hopes  of  moving  out  of  a  local  minimum.  The  paclmge  includes 
an  optional  eigenvector-based  starting  point  method  of  Rowan  and  Vannelli.  Together  with  the 
projection  algorithm  code,  the  graphics  package  provides  a  significant  step  towards  creating  a 
state-of-the-art  placement  package. 

Although  the  solutions  are  very  good,  the  performance  of  the  code  itself  is  significantly  less 
favorable.  The  current  implementation  of  the  algorithm  is  undesirably  memory  intensive  and 
extremely  slow.  This  is  largely  due  to  the  nature  of  the  algorithm;  solving  1000  least  squares 
problems  for  matrices  as  large  as  5000  by  400  can  be  time  consuming  (not  to  mention  the  other 
steps  taken  in  every  iteration).  However,  there  are  some  improvements  that  should  be  made,  such 
as  exploiting  sparsity  and  reducing  the  problem  size  by  considering  only  a  subset  of  the  possible 
constraints. 

Performance  notwithstanding,  the  overall  results  of  the  package  from  the  test  cases  considered 
to  date  axe  very  encouraging  and  merit  further  research  of  the  optimization  technique  for  problems 
of  these  types  as  well  as  more  work  on  the  package  itself. 

Current  methods  for  the  macro-cell  placement  problem  are  based  on  simulated  annealing  and 
the  Tabu  method.  In  many  cases,  the  problem  is  solved  by  hand  using  computer-aided  design 
tools  to  move  cells  on  a  display  screen.  This  is  the  first  time  that  a  projection  method  of  an  exact 
penalty  function  has  been  used.  For  every  problem  tested,  this  new  approach  produced  better 
quality  solutions.  However,  the  comparisons  were  done  on  standard  cell  problems  since  no  macro- 
cell  packages  were  available  at  the  time  the  testing  was  done.  Our  current  preliminary  package 
does  not  make  use  of  large  sparse  solvers  and  so  is  unnecessarily  computationally  expensive. 
However,  as  an  indication  of  possible  improvements,  we  implemented  a  version  that  computed 
the  search  direction  using  a  sparse  solver.  Memory  requirements  went  down  from  83  M  Bytes  of 
RAM  to  19  M  Bytes  and  the  new  version  was  over  thirty  times  faster. 

This  work  has  been  published  in  the  thesis  of  David  Mates  at  the  University  of  Waterloo, 
which  I  and  Dr.  VaneUi  directed  (see  Mates,  1993). 

4.10  Discontinuous  Problems 

This  research  involves  applying  piecewise  differentiable  techniques  to  discontinuous  problems  and 
was  the  main  body  of  research  carried  out  by  my  former  doctorate  student  Marcel  Mongeau,  now 
a  faculty  member  of  the  Paul  Sabatier  University,  Toulouse,  France.  The  problem  is  first  reduced 
to  an  unconstrained  one  using  pepalty  techniques.  The  objective  function  thus  has  its  domain 
partitioned  into  cells  over  each  of  which  jt  is  continuous.  Conditions  characterizing  a  minimum 
axe  developed.  In  the  first-order  algorithm,  descent  directions  are  obtained  by  using  an  active  set 
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approack.  Umvaiiata  mimnusatlon  io  aducred  via  a  specialized  Lae  searcL  which  recognizes  the 

deals  with  the  projection  of  the  Newton  direction.  The  numerical  linear  algebra  techniques  involve 
the  computation  of  orthogonal  and  aonorthogonal  projectors,  jmd  their  updates.  A  perturbation 
method  is  used  to  handle  degeneracy  and  the  approach  is  shown  to  be  practical. 

Applications  include  VLSI,  nonlinear  robust  regression  and  operation.?  researeh/iadustrial 
engineering. 


This  is  a  difBcult  area  of  some  importance  for  which  there  is  much  need  for  appLcable  algo¬ 
rithms.  Moreover,  the  area  bridges  certain  aspects  of  both  continuous  and  discrete  optimization. 

We  are  curicnlly  looking  at  a  fixed  charge  problem  appLed  to  assembly  fine,  production, 
origmally  considered  by  Hiraki  in  the  Japanese  OK  journal.  We  have  also  been  successful  at 
tackling  a  mixed  problem  involving  knapsack  constraints,  0-1  variables  reprecenting  allocations 
and  a  quadratic  model  for  the  generation  of  power.  This  latter  is  an  appHcatloa  originally 
investigated  by  Eric  IQein  of  Ontario  Hydro,  Canada  and  is  to  appear  in  EJOR..  Dr.  Mongeau  is 
maintaining  contact  with  Dr.  Klein  and  is  looking  at  more  general  problems.  He  is  also  working 
on  a  non-separable  fixed  charge  application  to  transportation  and  a  cutting  stock  problem  of 
Haberl,  that  was  pubLshed  in  JOTA  in  1991  and  has  recently,  been  generalized  in  a  subruissiou 


to  Mattcmatical  Progranamiag. 

Oui  algorithm  works  in  a  significantly  smaller  dimensional  space  than  the  approach  of  Hiraki, 
and  consequently  we  expect  to  be  able  to  tackle  much  larger  problems  than  he  is  able  to  handle. 
Rtithermore,  our  approach  is  more  general,  sine©  Hirali’s  method  requires  assumptions  that  are 
ra-tiier  artificial. 


TKese  results  appear  iji  Comi  aud  Mongeau  (1993a)  and  Conn  and  Mongeau  (1993b),  A 
synthesis  of  these  two  papers,  but  restricted  to  the  linear  case  is  to  be  piihlish^H  in  Conn  and 
Mongpj^.n  (1995)  The  application  to  discharge  allocation  for  hydro-electric  generating  stations  is 
most  interesting  in  that  we  are  able  to  concidcr  a  more  realistic  nonlinear  model  rather  than  the 
usual  linear  model.  In  particular^  it  is  known  that  in  the  opeiatlug  domain  (where  the  discharge 
is  sufficiently  large)  the  quantity  of  electricity  generated  by  one  particular  generating  unit  is  a 
quadratic  polynomial  function  of  the  water  drafted  through  the  unit. 


5  Technic^d  Publications 

Included  below  is  a  summary  of  the  twenty  technical  publications  to  date  that  have  benefitted 
from  and  acknowledge  the  ARPA  support.  1  will  send  copies  of  the  full  report r  n-nder  separate 
rover. 

l]  A.R.  Conn,  N.LM.  Gould  and  Ph.L.  Toint,  Large-Scale  Nonlinear  Constrained  Optimization 

During  the  past  ten  years,  much  progress  has  been  made  in  the  iheory  and  practice  of  con¬ 
strained  nonlinear  optimization.  However,  considerable  obstacles  appear  when  these  ideas  are 
applied  to  large-scale  problems,  Tiiis  is  important  as  many  real  applir.ationR  require  th#*  isolntion 
of  prohlftTUR  in  thouRAuds  of  unknowns.  In  some  areas,  in  particular  linoar  programming,  concid 
©rabk  progress  has  been  made.  But  even  modest  departures  into  nonlinearity,  for  example  the 
solution  of  large,  general  quadratic  pj:ogi<uiis,  prcsenl  cuusiderable  challenges.  This  is  apparent 
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when  one  views  the  paucity  of  software  for  solving  such  problems.  Unsurprisingly,  the  position 
does  not  improve  as  more  drastic  forms  of  nonlinearity  are  encountered. 

In  this  paper,  we  wUl  try  to  explain  why  the  difficulties  arise,  how  attempts  are  being  made 
to  overcome  them  and  what  the  problems  are  that  still  remain. 

(Published  in  ICIAM  ’91:  Proceedings  of  the  second  international  conference  on  industrial 
and  applied  mathematics,  Washington,  Ed  B..E.  O’Malley,  Jr.:  1992  pp.  51-70,  SIAM) 

2]  A.R.  Conn,  N.I.M.  Gould  and  Ph.L.  Toint,  Numerical  ^periments  with  the  LANCELOT 
package  (Release  A)  for  large-scale  nonlineair  optimization 

In  this  paper,  we  describe  the  algorithmic  options  of  Release  A  of  LANCELOT,  a  new  Fortran 
package  for  large-scale  nonlinear  optimization.  We  then  present  the  results  of  intensive  numerical 
tests  and  discuss  the  relative  merits  of  the  options.  The  experiments  described  involve  both 
academic  and  applied  problems.  Conclusions  are  finally  proposed,  both  specific  to  LANCELOT 
and  of  more  general  scope. 

(To  appear  in  Mathematical  Programming) 

3]  A.R.  Conn,  N.I.M.  Gould  and  Ph.L.  Toint,  Complete  numerical  results  for  large-scale  tests 
on  LANCELOT  Release  A 

This  report  contains  the  detailed  results  of  the  numerical  experiments  on  the  LANCELOT 
package  for  nonlinear  optimization  (Release  A).  These  results  constitute  the  basis  for  the  discus¬ 
sion  and  analysis  presented  in  the  report  [2]  above. 

4]  I.  Bongartz,  A.R.  Conn,  N.I.M.  Gould  and  Ph.L.  Toint,  CUTE  :  Constrained  and  uncon¬ 
strained  testing  environment  , 

The  purpose  of  this  paper  is  to  discuss  the  scope  and  functionality  of  a  versatile  environ¬ 
ment  for  testing  smzill  and  large-scale  nonlinear  optimization  algorithms.  Although  many  of 
these  facilities  were  originally  produced  by  the  authors  in  conjunction  with  the  software  package 
LANCELOT,  we  believe  that  they  will  be  useful  in  their  own  right  and  should  be  available  to 
researchers  for  their  development  of  optimization  software.  The  tools  are  av^able  by  anonymous 
ftp  from  a  number  of  sources  and  may,  in  many  cases,  be  installed  automatically. 

The  scope  of  a  major  collection  of  test  problems  written  in  the  standard  input  format  (SIF) 
used  by  the  LANCELOT  software  package  is  described.  Recognizing  that  most  software  was 
not  written  with  the  SIF  in  mind,  we  provide  tools  to  assist  in  building  an  interface  between 
this  inpnt  format  and  other  optimization  packages.  These  tools  already  provide  a  link  between 
the  SIF  and  an  number  of  existing  packages,  including  MINOS  and  OSL.  In  addition,  as  each 
problem  includes  a  specific  classification  that  is  designed  to  , be  useful  in  identifying  particular 
classes  of  problems,  facilities  are  provided  to  build  and  manage  a  database  of  this  information. 

(Published  in  ACM  Transactions  on  Mathematical  Software,  Volume  21,  ppl23-160,  1995) 

5]  A.R.  Conn,  N.I.M.  Gould  and  Ph.L.  Toint,  A  note  on  exploiting  structure  when  using  slack 
variables 

We  show  how  to  exploit  the  structure  inherent  in  the  linear  algebra  for  constrained  nonlinear 
optimization  problems  when  inequality  constraints  have  been  converted  to  equations  by  adding 
slack  variables.  * 

(Published  in  Mathematical  Programming,  Volume  67,  pp89-97,  1994) 
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6]  L  Bongartz,  P,H.  Calamai  and  A.R.  Conn  A  projection  method  for  Zy  norm  location- 
allocation  problems 

We  present  a  solution  method  for  location-allocation  problems  involving  the  norm,  where 
1  <  p  <  oo.  The  method  relaxes  the  0,1  constraints  on  the  allocations,  and  solves  for  both  the 
locations  and  allocations  simultaneously.  Nec^sary  and  sufficient  conditions  for  local  minima  of 
the  relaxed  problem  are  stated  and  used  to  devdop  an  iterative  algorithm.  This  algorithm  finds 
a  stationary  point  on  a  series  of  subspaces  defined  by  the  current  set  of  activities.  The  descent 
direction  is  a  projection  onto  the  current  subspace  of  a  direction  incorporating  second-order  infor¬ 
mation  for  the  locations,  and  first-order  information  for  the  allocations.  Under  mild  conditions, 
the  algorithm  finds  local  minima  with  0,1  allocations  and  exhibits  quadratic  convergence.  An 
implementation  that  exploits  the  special  structure  of  these  problems  to  dramatically  reduce  the 
computational  cost  of  the  required  numerical  linear  algebra  is  described.  Numerical  results  for 
thirty-six  test  problems  axe  included. 

(Published  in  Mathematical  Programming,  Volume  66,  pp'283-312, 1994) 

7]  A.R.  Conn,  N.I.M.  Gould,  A.  Sartenaer  and  Ph.L.  Toint  Global  Convergence  of  two  Aug¬ 
mented  Lagrangian  Algorithms  for  Optimization  with  a  Combination  of  General  Equality  and 

•  Linear  Constraints 

We  consider  the  global  convergence  properties  of  a  class  of  augmented  Lagrangian  methods 
for  solving  nonlinear  programming  problems.  In  the  proposed  method,  linear  constraints  are 
treated  separately  from  more  general  constraints.  Thus  only  the  latter  axe  combined  with  the  ob¬ 
jective  function  in  an  augmented  Lagrangian.  The  subproblem  then  consists  of  (approximately) 
minimizing  this  augmented  Lagrangian,  subject  to  the  linear  constraints.  In  this  paper,  we  prove 
the  global  convergence  of  the  sequence  of  iterates  generated  by  this  technique  to  a  first-order 
stationary  point  of  the  original  problem.  We  consider  various  stopping  r\ales  for  the  iterative 
solution  of  the  subproblem,  including  practical  tests  used  in  several  existing  packages  for  lin¬ 
early  constrained  optimization.  We  also  extend  our  results  to  the  case  where  the  augmented 
Lagrangian’s  definition  involves  several  distinct  penalty  parameters. 

8]  A.E..  Conn,  N.I.M.  Gould,  A.  Sartenaer  and  Ph.L.  Toint  Local  Convergence  Properties  of 
two  Augmented  Lagrangian  Algorithms  for  Optimization  with  a  Combination  of  General  Equality 
and  Linear  Constraints 

We  consider  the  local  convergence  properties  of  the  class  of  augmented  Lagrangian  methods 
for  solving  nonlinear  programming  problems  whose  global  convergence  properties  are  analyzed 
by  A.R.  Conn,  N.I.M.  Gould,  A,  Szurtenaer  and  Ph.L.  Toint  in  ’’Global  Convergence  of  two  Aug¬ 
mented  Lagrangian  Algorithms  for  Optimization  with  a  Combination  of  General  Equality  and 
Linear  Constraints”  In  these  methods,  linear  constreiints  are  treated  separately  from  more  general 
constraints.  These  latter  constraints  are  combined  with  the  objective  function  in  an  augmented 
Lagrangian  while  the  subproblem  then  consists  in  (approximately)  minimizing  this  augmented 
Lagrangian  subject  to  the  linear  constraints.  The  stopping  rule  that  we  consider  for  this  inner 

iteration  covers  practical  tests  used  in  several  existing  packages  for  linearly  constrained  opti- 

♦ 

mization.  Our  algorithmic  class  also  allows  several  distinct  penalty  parameters  associated  with 
different  subsets  of  general  equality  constraints.  In  this  pap^r,  we  analyze  the  local  convergence 
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of  the  sequence  of  iterates  generated  by  this  technique  and  prove  fast  linear  convergence  and 
boundedness  of  the  potentially  troublesome  penalty  parameters. 

9]  A.R.  Conn,  N.LM.  Gould,  A.  Saxtenaer  and  Ph.L.  Toint  Convergence  Properties  of  an 
Augmented  Lagrangian  Algorithms  for  Optimisation  with  a  Combination  of  General  Equality 
and  Linear  Constraints 

We  consider  the  ^obal  and  local  convergence  properties  of  a  class  of  augmented  Lagrangian 
methods  for  solving  nonlinear  programming  problems.  In  these  methods,  linear  and  more  gen¬ 
eral  constraints  are  handled  in  different  ways.  The  general  constraints  are  combined  with  the 
objective  function  in  an  augmented  Lagrangian.  The  iteration  consists  of  solving  a  sequence 
of  subproblems;  in  each  subproblem  the  augmented  Lagrangian  is  approximately  minimized  in 
the  region  defined  by  the  linear  constraints.  A  subproblem  is  terminated  as  soon  as  a  stopping 
condition  is  satisfied.  The  stopping  rules  that  we  consider  here  encompass  practical  tests  used  in 
several  existing  packages  for  linearly  constrained  optimization.  Our  algorithm  also  allows  differ¬ 
ent  penalty  parameters  to  be  associated  with  disjoint  subsets  of  the  general  constraints.  In  this 
paper,  we  analyze  the  convergence  of  the  sequence  of  iterates  generated  by  such  an  algorithm  and 
prove  global  and  fast  linear  convergence  as  well  as  showing  that  potentially  troublesome  penalty 
parameters  remain- bounded  away  from  zero.  . 

(To  appear  in  SIAM  J.  on  Optimization) 

10]  A.R.  Conn  and  M.  Mongeau,  Discontinuous  Piecewise  Differentiable  Optimization  I:  The¬ 
ory 

A  theoretical  framework  and  a  practical  algorithm  are  presented  to  solve  discontinuous  piece- 
wise  linear  optimization  problems.  A  penalty  approach  allows  one  to  consider  such  problems 
subject  to  a  wide  range  of  constraints  involving  piecewise  linear  functions.  Although  the  the¬ 
ory  is  expounded  in  detail  in  the  special  case  of  discontinuous  piecewise  linear  functions,  it  is 
straightforwardly  extendable,  using  standard  nonlinear  programming  techniques,  to  nonlinear 
(discontinuous  piecewise  differentiable)  functions. 

This  work  is  presented  in  two  parts.  We  introduce  the  theory  in  this  first  paper.  The 
descent  algorithm  which  is  elaborated  uses  active- set  and  projected  gradient  approaches.  It  is 
a  generalization  of  the  ideas  used  by  Conn  to  deal  with  nonsmoothness  in  the  lx  exact  penalty 
function,  and  it  is  based  on  the  notion  of  decomposition  of  a  function  into  a  smooth  and  a 
nonsmooth  part. 

In  an  accompanying  paper,  we  shall  tackle  constraints  via  a  penalty  approach,  we  shall  discuss 
the  degenerate  situation,  and  numerical  results  wih  be  presented. 

11]  A.R.  Conn  and  M.  Mongeau,  Discontinuous  Piecewise  Differentiable  Optimization  11: 
Degeneracy  and  Applications 

The  implementation  of  an  algorithm,  developed  in  ^’Discontinuous  Piecewise  Differentiable 
Optimization  I:  Theory"”  by  A.R.  Conn  and  M.  Mongeau,  to  solve  discontinuous  piecewise  linear 
optimization  problems  is  discussed,  and  limited  numerical  applications  are  presented. 

A  penalty  approach  allows  one  to  consider  such  problems  subject  to  a  wide  range  of  constraints 
involving  piecewise  linear  functions.  The  constrained  case  is  reduced  to  the  unconstrained  mini¬ 
mization  of  a  (piecewise  linear)  exact  penalty  function.  We  also  discuss  how  the  algorithm  is 
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modified  when  it  encounters  degenerate  points  and  other  points  where  the  objective  functiofi  is 
not  decomposable.  The  algorithm  is  applied  to  discontinuous  optimisation  problems  from  models 
in  industrial  engineering  and  manufacturing. 

12]  A-R-  Conn  and  M.  Mongeau,  Discontinuous  Piecewise  Linear  Optimization 

A  theoretical  framework  and  a  practical  algorithm  axe  presented  to  solve  discontinuous  piece- 
wise  linear  optimization  problems  dealing  with  functions  for  which  the  ridges  axe  known.  A 
penalty  approach  allows  one  to  consider  such  problems  subject  to  a  wide  range  of  constraints 
involving  piecewise  linear  functions.  Although  the  theory  is  expounded  in  detail  in  the  special 
case  of  discontinuous  piecewise  linear  functions,  it  is  straightforwardly  extendable,  using  stan¬ 
dard  nonlinear  programming  techniques,  to  nonlinear  (discontinuous  piecewise  differentiable) 
functions. 

The  descent  algorithm  which  is  elaborated  uses  active-set  and  projected  gradient  approaches. 
It  is  a  generalization  of  my  earlier  work  to  deal  with  nonsmoothness  in  the  li  exact  penalty 
function,  and  it  is  based  on  the  notion  of  decomposition  of  a  function  into  a  smooth  and  a 
nonsmooth  part.  A  penalty  approach  allows  one  to  consider  such  problems  subject  to  a  wide 
range  of  constradnts  involving  piecewise  linear  functions.  The  constrained  case  is  reduced  to  the 
unconstrained  minimization  of  a  (piecewise  linear)  li  exact  penalty  function.  We  also  discuss  how 
the  algorithm  is  modified  when  it  encounters  degenerate  points.  Preliminary  numerical  results 
are  presented:  the  algorithm  is  applied  to  discontinuous  optimization  problems  from  models  in 
manufacturing. 

(Submitted  to  Mathematical  Programming) 

13]  A.R.  Conn,  N.LM.  Gould  and  Ph.L.  Toint,  Improving  the  decomposition  of  partially 
separable  functions  in  the  context  of  large-scale  optimization:  a  first  approach. 

This  paper  examines  the  question  of  modifying  the  decomposition  of  a  partially  separable 
function  in  order  to  improve  computational  efficiency  of  large-scale  miiumization  algorithms  using 
a  conjugate-gradient  inner  iteration.  The  context  and  motivation  are  given  and  the  application 
of  a  simple  strategy  discussed  on  examples  extracted  from  the  CUTE  test  problem  collection. 

(PubHshed  in  "Large-Scale  Optimization:  State  of  the  Art”,  W.W.  Hager,  D.W.  Hearn  and 
P.M.  Paxdalos,  eds.,  Kluwer  Academic  Publishers,  pp82-94,  1994) 

14]  A.R.  Conn,  N.LM.  Gould,  A.  Sartenaer  and  Ph.  Toint  On  Iterated-Subspace  Minimization 
Methods  for  Nonlinear  Optimization 

This  paper  examines  a  class  of  Iterated-Subspace  Minimization  (ISM)  methods  for  solving 
large-scale  unconstrained  minimization  problems.  At  ea-ch  major  iteration  of  such  a  method,  a 
low- dimensional  manifold,  the  iterated  subspace,  is  constructed  and  an  approximate  minimizer 
of  the  objective  function  in  this  manifold  then  determined.  The  iterated  subspace  is  chosen 
to  contain  vectors  which  ensure  global  convergence  of  the  overall  scheme  and  may  also  contain 
conjugate  gradient  related  vectors  which  encourage  fast  asymptotic  convergence.  We  demonstrate 
that  this  approach  can  sometimes  be  very  advantageous  and  indicate  the  general  performance 
on  a  collection  of  large  problems.  Moreover,  comparisons  with  a  limited  memory  approach  and 
LANCELOT  are  made. 

(To  appear  in  "Proceedings  on  Linear  and  Nonlinear  Conjugate  Gradient-Related  Methods”, 
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L.  Adams  and  L.  Nazareth,  eds.,  SIAM,  1996) 

15]  A.R.  Conn,  N.LM.  Gould  and  Ph.L.  Toint,  Algorithms  for  laxge-scale  constrained  nonlin¬ 
ear  optimization  :  a  current  survey 

Much  progress  has  been  made  in  constrained  nonlinear  optimization  in  the  past  ten  years, 
but  most  large-scale  problems  still  represent  a  considerable  obstacle. 

In  this  survey  paper  we  attempt  to  give  an  overview  of  the  current  approaches,  including 
interior  and  exterior  methods  and  algorithms  based  upon  trust  regions  cind  line  searches.  In 
addition,  the  importance  of  software,  numerical  linear  algebra  and  testing  is  addressed.  We  try 
and  explain  why  the  difficulties  arise,  how  attempts  axe  being  made  to  overcome  them  and  some 
of  the  problems  that  still  remain. 

Although  there  is  some  emphasis  on  the  LANCELOT  and  CUTE  projects,  the  intention  is  to 
give  a  broad  picture  of  the  state-of-the-art. 

(Published  in  ’’Algorithms  for  continuous  optimization: the  state  of  the  art”,  E.  Spedicato, 
ed.,  pp287“-332,  Volume  434  of  NATO  ASI  Series  C:  Mathematical  and  Physical  Sciences,  Kiuwer 
Academic  Publishers,  Dordrecht,  The  Netherlands,  1994) 

16]  A.R.  Conn  and  Ph.L.  Toint,  An  algorithm  using  quadratic  interpolation  for  unconstrained 
derivative  free  optimization 

This  paper  explores  the  use  of  multivariate  interpolation  techniques  in  the  context  of  methods 
for  unconstrained  optimization  that  do  not  require  derivative  of  the  objective  function.  A  new 
algorithm  is  proposed  that  uses  quadratic  models  in  a  trust  region  framework.  The  algorithm  is 
constructed  to  require  few  evaluations  of  the  objective  function  and  is  designed  to  be  relatively 
insensitive  to  noise  in  the  objective  function  values.  Its  performance  is  analyzed  on  a  set  of  twenty 
examples,  both  with  and  without  noise. 

(To  appear  in  ”Nonlineax  Optimization  and  Applications”,  G.  Di  Pillo  and  F.  Giannessi,  eds., 
Plenum  Publishing,  1995) 

17]  A.R.  Conn,  N.LM.  Gould  and  Ph.L.  Toint,  A  globally  convergent  Lagrangian  barrier 
algorithm  for  optimization  with  general  inequality  constraints  and  simple  bounds 

In  this  manuscript  we  consider  the  global  and  local  convergence  properties  of  a  class  of  La¬ 
grangian  barrier  methods  for  solving  nonlinear  programming  ^problems.  In  such  methods,  simple 
bound  constraints  may  be  treated  separately  from  more  general  constraints.  The  objective  and 
general  constraint  functions  are  combined  in  a  Lagrangian  barrier  function.  A  sequence  of  such 
functions  are  approximately  minimized  within  the  domain  defined  by  the  simple  bounds.  Global 
convergence  of  the  sequence  of  generated  iterates  to  a  first-order  stationary  point  for  the  origi¬ 
nal  problem  is  established.  Furthermore,  possible  numerical  difficulties  associated  with  barrier 
function  methods  are  avoided  as  it  is  shown  that  a  potentially  troublesome  penalty  parameter 
is  bounded  away  from  zero.  This  paper  is  a  companion  to  our  previous  work  (see.  Conn  ei  nZ., 
1991)  on  augmented  Lagrangian  methods. 

(To  appeaLT  in  Mathematics  of  Computation,  1995) 

18]  B.  Reiser  and  B.  J.  Flehinger  and  A-  R-  Conn,  Estimating  component  defect  probability 
from  masked  system  success/failure  data 

We  consider  the  situation  where  a  system  consisting  of  k  components  wiU  fail  whenever  there 
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is  a  defect  in  one  or  more  of  the  components.  Due  to  cost  and  time  constraints  it  may  not 
always  be  feasible  to  learn  exactly  which  components  are  defective.  Instead,  test  procedures 
may  ascertain  that  the  defective  components  belong  to  some  subset  of  the  k  components.  This 
phenomenon  is  termed  masking.  Given  historical  masked  data  we  would  like  to  estimate  the 
quality  or  defect  probability  of  each  individual  component.  In  addition,  when  masked  data  will 
occur  in  the  future,  we  would  like  to  diagnose  the  defective  components.  In  this  manuscript  we 
discuss  the  analysis  of  such  data  and  suggest  a  two-stage  experimental  procedure  which  permits 
the  calculation  of  maximum  likelihood  estimates  of  the  probabilities  of  interest. 

(Submitted  to  IEEE  Transactions  on  Reliability) 

19]  A.R.  Conn,  N.I.M.  Gould  and  Ph.L.  Toint,  A  note  on  using  alternative  second-order 
models  for  the  subproblems  arising  in  barrier  function  methods  for  minimization 

Inequality  constrained  minimization  problems  are  often  solved  by  considering  a  sequence  of 
parameterized  barrier  functions.  Each  barrier  function  is  approximately  minimized  and  the  rel¬ 
evant  parameters  subsequently  adjusted.  It  is  common  for  the  estimated  solution  to  one  barrier 
function  problem  to  be  used  as  a  starting  estimate  for  the  next.  However,  this  has  unfortunate 
repercussions  for  the  standard  Newton-like  methods  applied  to  the  barrier  subproblem.  In  this 
paper,  we  consider  a  class  of  alternative  Newton  methods  which  attempt  to  avoid  such  difi- 
culties.  Such  schemes  have  already  proved  of  use  in  the  Harwell  Subroutine  Library  quadratic 
programming  codes  VE14  and  VE19. 

(Published  in  Numerische  Mathematik,  Volume  68,  ppl7-33, 1994) 

20]  A.R.  Conn,  N.I.M.  Gould,  A.  Sartenaer  and  Ph.L.  Toint,  Convergence  properties  of  min¬ 
imization  algorithms  for  convex  constraints  using  a  structured  trust  region 

In  this  paper,  we  present  a  class  of  trust  region  algorithms  for  minimization  problems  within 
convex  feasible  regions,  in  which  the  structure  of  the  problem  is  explicitly  used  in  the  definition 
of  the  trust  region.  This  development  is  intended  to  reflect  the  possibility  that  some  parts  of 
the  problem  may  be  more  accurately  modeled  than  others,  a  common  occurrence  in  large-scale 
nonlinear  applications.  After  describing  the  structured  trust  region  mechanism,  we  prove  global 
convergence  for  all  algorithms  in  oar  class. 

(To  appear  in  SIAM  J.  on  Optimization,  1995) 

6  FINANCIAL 

The  final  financial  statement  wiH  be  sent  under  separate  cover. 

6.1  EPILOGUE 

Our  general  objectives  in  this  research  have  been  twofold.  Our  primary  objective  has  always 
been  to  apply  optimization  to  real  problems  in  manufacturing.  Our  secondary  objective  is  to 
thereby  advance  research  into  nonlineair  programming  algorithms.  In  order  to  achieve  the  first 
objective  we  consider  it  essential^  to  work  directly  with  the  personnel  in  the  manufacturing  en¬ 
vironment.  We  actively  seek  out  manufacturing  groups  and  we  request  from  them  the  problems 
which  they  consider  important.  Thus  the  framework  in  which  we  work  consists  of  initially  famH- 
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iaimag  ourselves  witk  a  paxticulax  production  team,  foEowed  by  identifying  suitable  problems, 
in  consideration  of  both  the  appEcabiJity  of  our  expertise  and  the  significance  of  the  problem 
in  the  manufacturing  environment.  The  next  stage  consists  of  producing  an  initial  model  that 
is  simple  enough  to  give  rapid  feedback  (in  order  to  maintmn  and  stimulate  cooperation)  and 
realistic  enough  to  indicate  useful  results  (in  order  to  encourage  confidence  in  the  approach). 
This  stage  also  includes  the  coEection  of  data  from  the  practitioners.  Once  the  initial  model  has 
been  solved,  one  goes  through  a  cycle  of  analyzing  results,  expanding  and  modifying  the  model, 
re-solving,  collecting  new  data,  etc.  The  ultimate  goal  is  to  see  the  solution  implemented  in  the 
operation  of  the  manufacturing  site.  During  the  period  of  this  award  we  have  been  able  to  initiate 
a  considerable  number  of  projects  exactly  within  the  model  outlined  above. 

Given  the  current  progress  we  have  no  doubt  that  our  work  in  nonlinear  programming  can 
make  useful  contributions  to  manufacturing  and  we  intend  to  do  our  utmost  to  pursue  these 
approaches.  In  addition,  we  have  achieved  extremely  useful  input  and  support  for  our  more 
theoretical  endeavors. 

6.2  APPENDIX 

During  the  tenure  of  this  grant  my  coUeagues  N.  I.  M.  Gould,  Ph.  L.  Toint  and  myself  were 
awarded  the  Beale /  Orchard'Hays  Prize  (the  foremost  computational  award  in  mathematical  pro¬ 
gramming).  Since  a  substantial  amount  of  the  fundamental  research  carried  out  under  the  aus¬ 
pices  of  this  grant  involves  LANCELOT  and  CUTE  I  decided  to  include  the  entire  citation  in  an 
appendix. 


Beale/ Orchard-Hays  Prize  official  nomination 

Presented  at  the  15th  International  Mathematical  Programming  Symposium,  Ann  Arbor 

(USA),  15th  August  1994 

After  a  very  dose  competition  involving  many  exceEent  nominations,  the  Beale/Orchaxd-Hays 
prize  committee  is  pleased  to  announce  the  award  of  the  1994  prize  to  Andrew  R.  Conn  of  I.B.M., 
Yorktown  Heights,  Nicholas  I.M.  Gould  of  the  Appleton  Rutherford  Laboratory,  Oxfordshire  and 
Philippe  L.  Toint  of  the  Facdtes  Universitaires  Notre-Dame  de  la  Paix,  Namur  for  their  book 
entitled: 

LANCELOT:  A  Fortran  Package  for  Large-Scale  Nonlinear  Optimization,  Springer  Verlag, 
Berhn  1992. 

As  the  authors  say  in  their  introduction,  LANCELOT  (Large  And  Nonlinear  Constrained 
Extended  Lagran^an  Optimization  Techniques)  was  created  out  of  the  necessity  for  accurate 
modeling  of  physical,  sdentific,  statistical  and  economic  phenomena,  which  led  to  ever  larger  and 
more  challenging  nonlinear  optimization  problems.  However,  necessity  needs  to  be  recognized, 
and  complemented  by  the  courage,  determination,  and  ability  to  undertake  a  major  research 
effort  spread  over  several  years  and  nations. 

To  summarize  briefly  several  of  the  achievements  in  the  course  of  this  project: 
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1.  significant  work  on  the  structure  of  large  scale  nonlinear  optimization  problems,  and  on 
algorithmic  approaches  for  large  problems  i.e.  their  joint  paper  ”A  globally  convergent  aug¬ 
mented  Lagrangian  algorithm  for  optimization  with  general  constraints  and  simple  bonnds”, 

SIAM  Journal  on  Numerical  Analysis  28,  545-572  (1991). 

2.  the  development  of  a  uniform  input  representation  for  nonBneax  programs.  The  authors 
were  not  enamored  of  the  MPS  format,  but  were  convinced  by  the  user  community  that  if 
LANCELOT  were  to  get  the  widest  possible  use  on  real  problems,  total  compatibility  with 
the  mps  format  was  required.  Thus  they  deserve  commendation. 

3.  the  development  of  a  code  sufficiently  robust  that  it  efficiently  solves  without  tuning  the 
widest  range  of  nonlinear  programming  problems  of  any  code  yet  devised. 

4.  the  creation  and  distribution  of  the  CUTE  test  environment,  which  is  a  major  asset  to  the 
field. 

5.  the  quality  of  the  documentation  provided  by  the  book. 

6.  the  decision  to  distribute  LANCELOT  in  return  for  a  new  problem  from  the  user,  thus 
making  the  software  easily  available  and  simultaneously  promoting  research. 

In  conclusion  we  hope  and  believe  that  LANCELOT  is  but  a  beginning,  and  given  that  necessity 
has  received  such  a  helping  hand,  many  others  in  the  field  will  be  encouraged  to  tackle  the  many 
problems  crying  out  for  a  solution. 

Robert  R.  Meyer 
David  F.  Shanno 
Robert  Vaaderbei 
Laurence  A.  Wokey  (Chair) 
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